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LONG-TERM  GOALS 

In  recent  years  substantial  progress,  much  of  it  resulting  directly  from  ONR  funding  initiatives,  has 
been  made  in  understanding  fundamental  features  of  transport  and  mixing  in  oceans  using  methods 
derived  from  dynamical  systems  theory.  The  purpose  of  the  current  collaborative  research  is  to  extend 
these  methods  to  the  design  of  control  algorithms  for  Drifter  Motion  Planning  for  Optimal  Surveillance 
of  the  Ocean  (DRIMPOS).  This  effort  is  a  direct  attempt  to  transition  Lagrangian  based  dynamical 
systems  methods  from  diagnostic,  postdictive  tools  to  essential  and  active  components  in  the  design  of 
oceanographic  and  naval  observing  systems.  The  specific  goals  of  the  research  project  include  the 
development  of  flow-based  control  algorithms  for  drifting  autonomous  sensing  systems. 

OBJECTIVES 

Couple  dynamical  systems  ideas  and  control-theoretic  algorithms  to  produce  real-time  control  of 
gliders  based  on  the  output  from  high  resolution  coastal  ocean  model  forecasting  systems.  Specifically, 
use  knowledge  of  Lagrangian  ocean  dynamics  to  develop  readily  computable,  optimal  control 
algorithms  to  (1)  maximize  the  loitering  time  of  autonomous  surveillance  platforms  in  a  prescribed 
region  under  energy  constraints  (2)  minimize  the  distance,  over  an  extended  time  period,  between  a 
platform  and  a  specified  location  in  the  flow  (3)  optimize  sensor  coverage  of  a  given  surveillance 
region  by  single  or  multiple  platforms  and  (4)  perform  optimal  path  planning  for  an  AUV  platform  by 
minimizing  time  to  reach  any  number  of  specified  way-points  along  a  desired  observational  route. 

APPROACH 

Work  completed  under  this  grant  involves  close  collaboration  with  Igor  Mezic  at  the  University  of 
California  Santa  Barbara. 

We  study  high-level  path  planning  for  under-powered  vehicles  in  known,  spatially  complex,  time- 
varying  ocean  currents.  In  particular,  we  focus  on  the  problem  of  controlling  the  vehicle  from  its  initial 
position  to  a  (possibly  time-varying)  target  set  in  minimum  time.  Difficulties  arise  due  to  both  the 
abundance  of  locally  optimal  trajectories  and  the  lack  of  local  controllability  of  AUVs  in  the  presence 
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of  strong  currents.  Globally  optimal  trajectories  may,  however,  be  obtained  by  solving  a  dynamic  HJB 
(Hamilton  Jacobi  Bellman)  PDE  (partial  differential  equation)  for  the  time-varying  optimal  time-to-go 
function.  In  prior  work,  a  method  of  characteristics  known  as  the  "extremal  field"  method,  which  is 
equivalent  to  tracking  a  2D  "controllability  front"  backward  in  time  and  outward  in  space  from  the 
target  set  was  demonstrated.  Using  this  approach,  several  variants  of  the  control  problem  were 
successfully  solved  in  the  context  of  time  dependent,  two-dimensional  velocity  fields  produced  by  a 
data-assimilating  coastal  ocean  model  (NCOM  in  the  Adriatic  Sea). 

In  the  present  work,  we  exploit  a  special  property  of  minimum  time  control  to  obtain  the  same  globally 
optimal  trajectories,  far  more  efficiently.  This  is  accomplished  by  tracking,  for  any  potential  initial 
AUY  position,  a  one-dimensional  "reachability  front"  forward  in  time.  The  utility  of  the  approach  is 
demonstrated  for  the  application  of  approximate  AUV  trajectory  tracking  in  ocean  flows.  In  this 
context,  optimal  trajectories  computed  using  the  newly  developed  forward  in  time,  open-loop 
algorithm  compare  very  favorably  with  those  previously  computed  by  the  extremal  field  method. 

Importantly,  reformulation  of  the  control  problem  to  a  forward-in-time  approach  allows  for  the 
efficient  computation  of  globally  optimal  trajectories  in  current  fields  of  the  fonn  u(x,y,t)  := 
u(x,y,z(t),t).  Given  the  disparity  of  vertical  and  horizontal  current  speeds  in  the  ocean  and  the  dive 
profiles  of  typical  AUYs,  the  vertical  speed  of  the  AUV  will  dominate  ambient  vertical  velocities 
almost  everywhere.  As  such,  the  next  level  of  investigation  will  concentrate  on  the  role  of  vertical 
shear  for  AUV  control  in  strong  horizontal  flow  fields  where  the  goal  is  optimal  control  in  the 
horizontal  for  a  vehicle  with  prescribed  dive  profile  z(t).  The  reduction  in  dimensionality  afforded  by 
the  forward  in  time  approach  allows  feasible  computations  of  globally  optimal  trajectories  for  AUV 
path  planning  in  3D,  time-dependent  horizontal  flow  fields. 


Backward:  Controllability  Front 


Forward:  Reachability  Front 


Figurel:  Schematic  of  the  two  approaches  to  computing  optimal  trajectories  in  time-varying  flows. 
Left  gives  feedback  control  by  computing,  for  each  final  state,  the  arrival  time,  T(t),  backwards  in 
time.  Right  panel  shows  the  open  loop  approach  of  computing,  for  any  position  at  the  initial  time, 

the  reachability  front  forward  in  time. 

The  differences  between  the  two  approaches  to  computing  globally  optimal  trajectories  are  shown 
succinctly  in  Figure  1.  Previous  funding  resulted  in  reference  [1]  where  we  characterized  solutions  of 
minimum  time  HJB  equations  for  which  the  value  function  is  (a)  discontinuous  and  (b)  dynamic,  using 
a  novel  implementation  of  the  extremal  field  approach.  In  particular,  we  did  this  for  minimum  time 
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feedback  control  in  a  time-varying  flow  fields  produced  by  coastal  ocean  models  (see  Figure  2). 
Moreover,  we  re-interpreted  the  extremal  field  approach  as  a  marker  particle  method  of  tracking  the 
same  fronts  tracked  by  Ordered  Upwind  Methods,  but  which  allows  incremental  refinement  of  the 
solution  and  is  inherently  easy  to  parallelize.  To  be  precise,  computing  the  extremals  by  integrating  the 
Euler  Lagrange  equations  backward  in  time  is  equivalent  to  tracking  a  2D  "controllability  front" 
backward  in  time  and  outward  in  space  from  the  target  set  as  shown  in  the  left  panel  of  Figure  1.  Such 
computations  provide,  at  any  point  in  space-time,  the  ‘time-to-go’  function.  Optimal  trajectories  move 
down  the  gradient  of  this  function. 

Given  the  relative  simplicity  of  the  cost  functional  for  minimum  time  control,  precisely  the  same 
globally  optimal  trajectories  can  be  obtained  more  efficiently  by  tracking  a  ID  "reachability  front” 
forward  in  time.  The  resulting  continuum  in  space  time  of  fronts  (or,  equivalently,  extremals)  provides 
only  a  single  slice  of  the  solution  of  the  HJB  equation,  and,  in  particular,  a  HJB  equation  for  the  arrival 
time  function,  not  the  time-to-go  function.  While  this  implies  open-loop  control,  the  optimal 
trajectories  are  identical  and  the  reduction  in  dimensionality  of  the  surface  along  with  the  ability  to 
both  efficiently  re -mesh  and  trim  globally  suboptimal  trajectories  from  this  surface  leads  to  large 
improvements  in  efficiency. 

WORK  COMPLETED 

1)  An  efficient  and  accurate  means  of  computing  globally  optimal  trajectories  using  a  forward 
in  time  approach  has  been  developed.  Results  of  the  algorithm  compare  favorably  with 
previously  developed  (and  more  computationally  intensive)  backward  in  time  optimal  control 
approaches.  These  comparisons  have  been  done  in  the  context  of  AUV  control  problems  in 
time-varying  flow  fields  produced  by  data-assimilating  coastal  ocean  models.  A  manuscript 
detailing  the  application  of  the  control  algorithm  to  AUV/glider  control  in  realistic  ocean 
models  is  nearing  completion  for  submission  to  IEEE  journal  of  Ocean  Engineering. 

2)  Preliminary  work  on  the  application  of  the  forward  in  time  control  algorithm  to  horizontal 
vectors  fields  varying  in  3  spatial  dimensions  has  begun. 

RESULTS 

Results  detailing  the  application  and  efficiency  of  the  control  algorithm  for  maneuvering  autonomous 
sensor  platforms  for  maximum  loitering  time  in  extended  (0(20  x  20  km))  regions  were  considered 
previously.  We  showed,  in  the  context  of  a  well  documented  NRL  coastal  model  of  the  Adriatic  Sea, 
that  the  optimal  control  algorithm  can  provide  an  order  of  magnitude  increase  in  the  residence  time  of 
a  glider  in  such  a  region  when  compared  to  simple,  a-priori  path  planning  strategies.  Our  goal  here 
has  been  to  develop  the  forward  in  time  algorithm  and  demonstrate  its  efficiency  and  accuracy.  In  the 
following,  we  consider  the  simplest  case  of  optimal  path  planning  from  an  extended  set  of  potential 
initial  conditions  to  a  single  prescribed,  fixed  in  time,  final  destination.  More  general  situations  such  a 
station  keeping,  control  to  nominal  (Lagrangian)  trajectories,  and  way  point  tracking  may  be  dealt  with 
in  an  entirely  analogous  manner. 

Comparison  of  Approaches :  Figure  2  shows  the  path  planning  results  computed  using  the  full, 
backward  in  time  extremal  field  approach  in  the  context  of  the  NCOM  Adriatic  flow  field.  The  target 
destination  is  indicated  by  a  fixed  white  dot  located  at  (190,270)  in  model  grid  units.  Each  grid  cell  is 
approximately  lxl  km.  Four  sample  globally  optimal  trajectories  are  shown  along  with  the  evolving 
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time-to-go  surface  computed  by  backward  in  time  integration,  over  all  possible  arrival  times,  from  the 
final  destination.  The  horizontal  extent  of  the  colored  surface  indicates  the  24  hour  controllability  front 
of  the  target  point  -  the  set  of  all  points  in  the  domai  which  can  be  controlled  to  (190,  270)  within  24 
hours  given  that  the  maximum  speed  of  the  AUV  is  0.9  km/h.  The  panel  to  the  lower  left  gives  the 
heading  relative  to  the  direction  of  the  target.  Interestingly,  globally  optimal  trajectories  rarely  head 
directly  to  the  target  location  but  instead  make  use  of  the  time  dependent  currents.  The  panel  to  the 
lower  right  shows  the  magnitude  of  the  model  currents  along  each  trajectory  indicating  the  ability  of 
the  algorithm  to  identify  globally  optimal  trajectories  and  conduct  path  planning  even  in  strong  flows. 
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Figure  2:  Top  panel:  Slices  of  time-to-go  function,  T(x,t),  and  four  simulated  closed  loop 
trajectories  for  the  time  varying  NCOM  Adriatic  flow .  Left  bottom  panel:  Heading  relative  to  the 
target  location  of  the  four  controlled  A  U  V  Right  bottom:  Instantaneous  magnitude  of  the  flow 
speed  along  the  four  trajectories.  The  maximum  AUV  speed  is  s  =  0.9. 
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Figure  2  shows  a  comparison  of  the  forward  and  backward  in  time  approaches  to  the  same  control 
problem.  In  each  time  slice,  the  reachability  front  (shown  in  thin  black)  of  the  trajectory  initialized  at 
(188,  261)  is  superimposed  upon  the  previously  computed  time-to-go  surface.  Comparison  of  the 
optimal  trajectory  for  the  full  algorithm  (black  dots)  to  the  open-loop,  forward  in  time  optimal 
trajectory  (black  line)  indicates  that  the  two  approaches  produce  identical  final  results. 


Figure  3:  Snapshots  of  the  computed  forward  in  time  reachability  front  (in  thin  black)  superimposed 
on  the  previously  computed  time-to-go  function  for  an  IS  COM  Adriatic  flow  field.  A  single  optimal 
trajectory  computed  using  the  open-loop,  forward-in-time  algorithm  (solid  line)  is  compared  to  an 
optimal  trajectory  computed  using  the  closed-loop,  backward  in  time  approach  (solid  circles).  The 

results  of  the  two  approaches  are  indistinguishable. 

The  efficiency  of  the  forward  in  time  approach  for  path  planning  from  a  given  initial  positions  is 
shown  in  Figure  3  where  a  different  time  period  of  the  NCOM  data  was  used  and  the  final  position  was 
fixed  at  (185,  255)  in  local  grid  units.  This  location  is  in  a  bathymetrically  complex,  eddying  region 
near  the  model  eastern  (Croatian)  coastline.  The  arrival  time  function  is  shown  along  with  6  sample 
globally  optimal  trajectories.  The  45  hour  reachability  front  is  given  by  the  last  countour.  The 
algorithmic  efficiency  derives  from  the  ability  to  readily  clip  globally  suboptimal  trajectories  from  the 
computed  one-dimensional  reachability  front.  Such  suboptimal  trajectories  necessarily  occur  at  self¬ 
intersection  points  (cusps/shocks)  in  the  front. 
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Figure  4:  Example  of  globally  optimally  controlled  trajectories  computed  in  N COM  Adriatic  model 
field.  Shown  are  the  arrival  time  function,  the  (discontinuous)  reachability  front  and  several  optimal 
trajectories.  Positions  are  relative  to  the  model  grid  with  the  model  Croatian  coastline  shown  in 

black  on  the  right. 


IMPACT/APPLICATIONS 

The  research  extends  the  application  of  rigorous  and  objective  optimal  control  theory  to  a  number  of 
practical  AUV/glider  operational  strategies.  The  algorithms  developed  are  explicitly  suited  to 
computing  optimal  control  in  the  context  of  highly  nonlinear,  time-dependent  numerical  vector  fields 
derived  from  ocean  model  output.  Gains  in  computational  efficiency  provided  by  a  forward  in  time, 
open-loop  approach  will  allow  for  direct  investigation  of  path  planning  for  AUV/gliders  executing 
prescribed  dive  protocols  (explicit  prescription  of  z(t))  in  complex,  time  dependent,  3-dimensional 
current  fields  given  by  model  output. 

RELATED  PROJECTS 

This  work  is  collaboration  under  the  ONR  PLUS  effort  with  Prof.  I.  Mezic'  and  his  group  at  the 
University  of  California  Santa  Barbara.  The  PI  actively  collaborates  with  Professors  T.  Ozgokmen  and 
A.  Griffa  at  the  University  of  Miami  on  ONR  funded  research  concerning  Lagrangian  data 
assimilation,  optimal  deployment  strategies,  model-data  intercomparison  and  the  sensitivity  of 
Lagrangian  Coherent  Structure  boundaries  to  model  error  and  filtering.  Similarly,  the  PI  continues 
regular  collaboration  with  Professors  A.D.  Kirwan  and  B.  Lipphardt  at  the  University  of  Delaware.  All 
named  collaborators  are  active  participants  in  the  ONR  Multiple  University  Research  Initiative: 
DYNAMICAL  SYSTEMS  THEORY  IN  4D  GEOPHYSICAL  FLUID  DYNAMICS  funded  in  2010. 
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